DOCUMENT RESUME 



ED 040 578 



88 



EM 008 004 



AUTHOR 

TITLE 



INSTITUTION 
SPONS AGENCY 

PUB DATE 
CONTRACT 
NOTE 



Cook, J. Marvin 

An Experimental Research Design for Comparing the 
Retention Effects of Different Instructional 
Treatments. An Occasional Paper. 

Maryland Univ. , Baltimore. 

Office of Education (DHEW) , Washington, D.C. Bureau 
of Research. 

Feb 70 

OEC-3-9-090018-0021 (010) 

28p. 



EDRS PRICE EDRS Price MF-$0. 25 HOS1.50 

DESCRIPTORS ^Comparative Analysis, Educational Research, *Post 

Testing, ^Retention, Statistical Analysis, Teaching 
Methods 



ABSTRACT 

The effects of instructional treatments are most 
often evaluated on the basis of an analysis of posttests given 
immediately following the completion of the instruction. In order to 
compare the relative effectiveness of different methods of 
instruction, it is also useful to know something about the retention 
effects of various instructional treatments. Interpretation of the 
effect different teaching methods have on retention must be based, 
not merely on performance on the delayed posttest, but on the 
relationship of the performances on the immediate and the delayed 
posttests. An example of research involving four instructional 
treatments and two repeated measures (posttests) with no pretests is 
discussed as an application of repeated measures analysis to the 
study of retention. The equations necessary for determining certain 
critical factors in the analysis are demonstrated. The location of 
significant differences, calculatio n of the slopes of retention 
curves, and tests for assumptions of homogeneity of variance are also 
illustrated. A reference list is appended. (JY) 
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Comparing the Retention Effects of 
Different Instructional Treatments 



The effects of instructional treatments are most often evaluated on 
the basis of an analysis of posttests given immediately following the 
completion of the instruction* In such analyses a comparison of treatment 
means is made to determine the relative value of the treatments in 
facilitating the amount of learning or the speed of learning. When the 
effect on learning is .the point of interests elementary ANOVA research 
designs may be utilized to compare the relative effectiveness of different 
methods of instruction. However, when there is an interest in investigating 
the retention effects of instructional treatments, the problem of analyses 
becomes more complex. 

This paper describes in Part I some interpretative problems involved 
in the investigation of such retention effects. In Part II the use of 
the repeated measures analysis for studying retention is demonstrated. For 
the purpose of this paper the following definitions will be utilized: 



Subject matter is defined in a broad sense as whatever 
it is you, the teacher, are intending to achieve in 
the way of educational intents. The educational 
intents might include among others, any combination 
of: knowledge, problem solving skill, creativity, 
or master skills. 

Retention is defined in terms of the amount of learned 
subject matter that' is retained after a specified 
interval of time. 

Rate of forgetting is defined in terms of the change in 
the scores made on the immediate posttest and on the. 
delayed posttest over a specified interval of time. 

• . • It : 

Over-all f performance produced by a treatment is defined 
• in terms of the score obtained when the scores made 
on the two or more posttests, by the group of students 
und^r a particular treatment: are pooled. 



/ / 
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Part I 

In terms of learning and retention there are several possible effects 
that might be produced by an instructional treatment* Although an 
instructional modification might be cf little value in increasing the amount 
learned or the speed of learning* it might better facilitate retention* 

That is* while an instructional treatment might produce relatively 
inefficient learning* it might produce greater resistance to forgetting 
than some other methods* The opposite may also occur: An instructional 
treatment might facilitate the amount learned but provide very little 
resistance to forgetting* While producing differences in over-all 
performance* treatments may or may not produce different rates of 
forgetting* As previously mentioned* the over-all performance produced 
by an Instructional treatment will be dependent upon its effect on 
resistance to forgetting* 

Educational investigators wishing to compare the effects of different 
instructional treatments on learning and retention commonly use a 
modified form of the pre-posttest research design* Students are randomly 
assigned to two or more treatments* Depending upon whether or not the 
subject matter to be taught is familiar to the stud&£ts* a pre-test may 
or may not be administered to them before beginning the experimental 
treatment* After the experimental treatments are administered* achievement 
tests (henceforth referred to as posttests in this paper) are than given 
to measure the performance level of the students* If the investigator is 
only Interested in the effects produced by the treatments on learning* 
then one posttest may be given immediately after the Instructional treatment, 
If the investigator is concerned with retention over time* then the 
immediate posttest may be administered plus one or more additional delayed 
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posttests administered at successive time intervals, 

A discussion of three possible outcomes from a comparison of two 
instructional treatments will delineate the factors which must be considered 
when interpreting the comparative effects of treatments on retention. 

For simplicity and clarity this discussion will be limited to a two- 
treatment, two-posttest situation where the two posttests are separated 
by an interval of time. Figures 1, 2, and 3 contain retention curves (i,e,, 
forgetting curves) obtained by plotting the data from the two successive 
post-test measures. These curves provide a visual indication of the rate 
of change in scores from the immediate posttest to the delayed posttest. 

Outcome 1 

Figure 1 is a plot of the tabulated data shown in Table 1, The data 
tabulated are the mean performance scores of the groups of students 
subjected to the treatments. There is statistically no difference between 
the effects of T^ and T 2 on performance on the immediate posttest. However, 
T- produced a lower performance on the delayed posttest than T 2 * Hence, the 

JL 

students subjected to T^ reflect a higher rate of forgetting than the 
students subjected to T^, 

Summarizing, there is no significant difference between 
the effect of the two treatments on learning; however, 

T 2 facilitates retention more than T^. 

There is a decrease of 12,5 from the mean score of immediate posttests 
to the mean scores .of the delayed posttests (from 40,5 to 28), This pattern 
of loss does not hold for each of the two treatments. For T^, the mean 
score decreases 20 from the immediate to the delayed posttests (from 40 to 
20). For T 2 > the score decreases only 5 (from 41 to 36), Since the over- 
all difference between performance on the Immediate posttest and performance 
on the delayed posttest is not identically reflected by each treatment, 
there is an interaction between the treatments and the measures of retention 
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Treatments 


Xosnediate 


Table 1 

Mean Performance 
Delayed 


Over-all 

Performance 

Score 


*i 


40 


20 


30 


*2 


41 


36 


38.5 


Means of posttests 


40.5 


28 





over time* The difference in the slopes of the retention curves in Figure 

1 reflects the existence of the interaction. Figure 1 also reveals by 

the relative slope of the two curves that treatment T 2 produces greater 

resistance to forgetting than T^. 

Note that T and T not only produced differences in over-all 
1 2 

performance* but also produced different rates of forgetting. 



Mean 

Performance 





Figure 1 



Outcome 2 



Posttests of learning and retention are usually highly correlated 
because they must measure performance on the same subject matter. The 
problem of interpreting the scores on delayed retention posttests is a 
result of this fact. If two treatments yield differences in scores on the 
immediate posttest (reflecting a difference in amount learned), it 
could be expected because of the posttest correlation that the differences 
in scores would still exist on the delayed retention posttest. Misleading 

conclusions could be obtained if this high correlation between the learning 

* 

and the retention postt^sts were Ignored. The data from possible Outcome 
2 illustrates the Interpretive problem. 

The tabulated data in Table 2 is plotted in Figure 2. There is a 

significant difference between the performance on the posttest of those 

students subjected to and the performance on the same test of the 

students subjected to This difference in performance is retained on 

the delayed posttest. Although the students who were taught under T^ 

tested higher on both the immediate posttest and the delayed post test, the 

fact that the forgetting curves of T^ and T 2 are parallel (i.e., slopes 

are the same) indicate that the treatments did not produce different rates 

of forgetting. Hence, the existent differences on the delayed po3ttest is 

attributable to the differing effect of the treatments on the amount 

learned and not to a differing effect on retent ion 0 

Summarizing, T« facilitated learning more than T 2 
but apparent lynad no advantage over T 2 in terms 
of retention. 




} 
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Table 2 



Treatments 


Mean Performance 
Immediate Delayed 


Over-all 

Performance 

Score 




60 


50 


55 


*2 


45 


35 


40 


Means of posttests 


52.5 


42*5 





Mean 

Performance 




Figure 2 

Note that the decrease of 10 from the mean score on the immediate 
postte8t8 to the mean score of the delayed posttests (i.e,, from 52 .5 to 
42*5) is identically reflected by each treatment* Hence, there is no 
interaction between the treatments and the measures of retention over time* 
The parallel retention curves in Figure 2 reflect this condition* 

It is alto worthwhile to note that while T^ and produce differences 
in over-all performance, the treatments did not produce different rates 
of forgetting* 

Outcome 3 



An interesting and important question for instructional mddification 
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is whether an instructional treatment facilitates retention independent of 
its effect on learning. The data from possible Outcome 3 lends itself to 
a consideration of this question. 

The data of a third possible outcome is tabulated in Table 3 and 
plotted in Figure 3. The data reveals a difference between the performance 
of students under T^ and the performance of students under T 2 on the 
immediate posttest. However, a difference is essentially non-existent on 
the delayed posttest. The data chows that the treatments produced 
differences. in learning, but that the differences were cancelled over time. 
Hence, the apparent advantage that T^ illustrates over in learning is 
wiped out when the two treatments are compared on the basis of the delayed 
posttest. 

But as we have seen in our discussion of the previous two possible 

outcomes, it is not sufficient to evaluate treatments just on the basis 

of the scores made on delayed posttests. 

The relation of these two sets of scores must 
be first determined for a proper evaluation 
to be made. 

Note that the decrease of 15.5 from the mean score of the immediate 
posttests to. the mean score of the delayed posttests (i.e,, from 55 to 
39.5) is not identically reflected on each of the two treatments. 
Therefore, there is an interaction between the treatments and the measures 
of retention over time. Figure 3 reflects this interaction by showing the 
differences in slopes of retention curves. In this outcome T^ produced a 
relatively lower rate of forgetting than T 2 * 



o 
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Table 3 





Mean Performance 


Over-all 


Treatments 


Immediate 


Delayed 


Performance 

Score 


T i 


45 


39 


42 


T 2 


65 


41 


53 


Means of posttests 55 


39.5 





As In Outcome 1 9 the data in Outcome 3 reveals that a difference 
existed not only between the over-all performances produced by and T 
but also between the rates of forgetting produced by T- and T« # 

X 4 



Mean 

Performance 

i 



Figure 3 

Conclusion 

These three possible effects illustrate the point that interpretation 
. of the effect different teaching methods have on retention must be based 
not merely on performance on the delayed posttest (i.e., the retention 
measure) but on the relationship of the performances on the immediate 
and the delayed post tests* 

o 

ERIC 




20 . 



Immediate Delayed 

Posttests 
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It is the rate of forgetting that is the key 
criteria for determining whether or not a 
particular treatment facilitates retention more 
than another treatment* 

The existence of a statistically significant treatment-by-retention measure 
interaction indicates that there is a difference in the rate of forgetting 
produced by the treatments* A plot of the retention curves reveals which 
treatment produces the lower rate of forgetting* 



Part II 

The thrust of the discussion in Part I was that the analysis of 
retention data is basically a problem of comparing the rates of forgetting 
curves* Wodtke (1) states that the repeated measures design seems most 
appropriate for the study of differential rates of forgetting* Excellent 
descriptions of repeated measures design have been given by Grant (2) and 
Winer (3) *A repeated measures analysis provides evidence concerning the 
general superiority in terms of over-all performance between treatments 
and it also provides tests for the differences in the slopes of the 
forgetting curves* An analysis of the general superiority between treatments 
is based upon statistical differences between the over-all performance 
produced by each instructional treatment* 

For the purposes of demonstrating the application of the repeated 
measures analysis to studying retention, an example research Involving 
four instructional treatments and two repeated measures (posttests) with 
no pre-test will be discussed* Note that only one delayed posttest will 
be utilized. If more than one delayed posttest were to be administered, 
a trend analysis could be performed to determine the shape of the forgetting 
curves* A discussion of such an analysis is given by Grant (2)* 

Winer (3) describes a repeated measures analysis design appropriate for a 
study which includes a pre-test and two or more posttests. 
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Example 

A researcher wished to test the following two research hypotheses: 

1) Four instructional treatments Al, A2, A3, and A4 have 
differential effects on the over-all performance of 
students, 

2) The four treatments produce different rates of forgetting. 

He classified the 72 students in the fourth-grade classes of the ADC 
Elementary School into three intelligence levels - high, middle, and low. 
The students who scored above 120 on the Wise , between 100 and 120 on 
the Wise, and below 100 on the Wise were respectively identified as high, 
middle and low. The students from each of the ability levels were then 
randomly assigned to the three treatments. While the n f s in each sub-level 
were not equal, they were not inordinately low. The layout for this 
study appears as follows: 

posttests 

Immediate Delayed 

Treatment Al Hi X 

Ml 

LI 



Treatment A2 H2 

M2 

L2 

Treatment A3 H3 

M3 

L3 

Treatment A4 H4 

M4 

14 




a cell 



Figure 4 



The X shown in the layout is an example of datum entered into a 
particular location in the layout, X located as shown would be the mean 
score of the students of high ability level (HI) under treatment Al on 
the immediate posttest. 
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In 'this design, an individual’s score is assumed to be represented a s 
follows : 



Jkp 
where : 






Kf» * \k + 



+ £ ^p 



.th 



th 



X jkp is the score found ^der the j Uii posttest; in the k un treatment group; 
and in the 1^ ability level, 
is the population mean. 

til 

OC^ is the effect of being in the k treatment group. This effect in 
this layout is estimated by 



O t 



A1 



4 



* X A1 X T 

- Xa, -X 



°^A3 ~ X A3 ~ X 



- X, 



- X. 



'A2 " *AZ ~ A T <*-A4 ~ "A4 

is the effect of the performance measure being posttest j . It is 

estimated in this example by subtracting each posttest mean from 



the grand mean: 

A 



Tmtfntd. T 



and 



S X_ -X. 






th 






rM. ’Pel. ’T 
is the effect of being in the p uu level of ability within treatment k. 
This effect is estimated by subtracting each level mean within a 
treatment from its treatment mean. 

is the interaction of treatments and repeated performance measures 
(posttests) over time. It is estimated in this example layout by 
finding the mean score of all the scores in a cell consisting of 
a specific treatment group and a particular posttest. The dotted 
lines in the layout define such a cell. The cell effect is 
estimated by subtracting the grand mean from this mean score. 

The alpha effect and the beta for this cell are 
also subtracted out and the resulting number is the estimated 

A 

interaction effect .. . 

Ok 



V 
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A.e. 




is the interaction 



('Xjk ~ * 7 ^ " °*IC 

s <_*« "/r> ** -_* t ) - " X T ) 

*j.k “ ** " ^ + 

of levels of ability and performance on posttests 



within a particular treatment. The interaction effect can be 



estimated for each datum in the analysis by the following formula: 




^ ^jUp) 

is the error term. 





The assumptions made for the repeated measure analysis were: 

' 1. The students from ^ach of the ability levels were randomly assigned 
to the four treatments. 

2. The variances of the ability level means within the various treatments 
are homogeneous • 

3* The variance of the "interaction of posttests and ability levels within 
treatments" is a pooling of the variance of the scores in each treat- 
ment group about the treatment mean after the effects of posttests 
and the effects of ability levels within treatments have been sub- 
tracted. These variances for each treatment group are homogeneous. 

4. The data collected satisfies at least the interval scale. 

5. The scores collected as data are normally distributed. 

The research was designed to test the following two statistical hypotheses: 

For row effects: 



Null: at. = 0, for all k 

Alternate: f 0, for some k 

For interaction effects: 



Null: 7T.. = 0, 

Alternate: Tf.*, ^ 0, 

r 



for all jk 
for some 



Findings and Analysis ; 



Posttests \ 



Treatments 


Levels 


Immediate 


Delayed 


Total 






HI 


45 


21 


66 




A1 


Ml 


, \ 40 


15. 


55 


150 




LI 


' f i V 25 


4, 


29 






H2 


87 


78 


165 




A2 


M2 


77 


67 


144 


413 




L2 


58 


46 


104 






H3 


49 


19 


68 




A3 


M3 


46 


27 


73 


182 




L3 


30 


11 


41 




• 


H4 


96 


93 


189 




A4 


m 


88 


83 


171 


494 




14 


70 


64 


134 








711 


£ X D = 528 


1239 




1, Total sum of 


squares 


(SST) 










SST 


» £x 2 - (£x) 2 












N 








* 




*82,605 - (1239) 2 


- 82,605 - 


63,735 





*18,870 



2 . 



Sun of squares between ability levels within all treatments (the variability 
between ability level means and the grand mean). 



SSBetween levels 



(£ x H1) 2 + 

HR 

HI 



(^ X M1) 2 * 

n 

Ml 



+ x t4) 2 - (£X) 2 



n 

14 



T 



(66) 2 -J* (55) 2 (134) 2 - 

T ~ 2 ~ 



- 



63,735 



14- 



3. Sum of squares between treatments (the variability between treatment means 
and the grand mean) , 

SSBetween treatments = ^ X Al} 2 + . . . + ^ X A4^ 2 - (^X) 2 

n_ n . N 



A1 



AA 



63,735 



- (150) 2 + . . . * (494) 2 - 
6 6 
® 14,636.5 

4* Sum of squares for levels of ability within treatments (the variability of 
ability level means from treatment means). 

SSLevels within treatments s SSBetween ability levels - SSBetween 

treatments 

® 17,040.5 - 14,636.5 
s 2,404.0 

5. Sum of squares within ability levels (variability of ability level scores 
from ability level mean). 

SSWithin levels » SST - SSBetween ability levels 

» 13,070.0 - 17,040.5 
« 1,829.5 
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6. Sum of squares for posttests (variability of posttest means from the grand 
mean) . 



SSP 



(iXj) 2 * 


(*x D ) 


n 


n 


I 


D 


M 

to 

+ 


(523) 2 


12 


“T2 


1,623.3 





N 



- 63,735 



7. Sum of squares for cells (variability of cell means from the grand mean). 
The quantity is not used in the summary table; if: will be used to find 
interaction sum of squares. 

SSCells » ^ X A1I) 2 * ^ X A1D) 2 ♦ ^ X A2lV>...+ / £X A4D) 2 - (*X) 2 

n . « m n . tf _ r . n . < N 

All AID A2I A4D 

» (110) 2 + (40) 2 ♦ (222) 2 + ... + (240) 2 - 63,735 

3 3~ ~ 3 ~ 

- 16,4X6.7 
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8. Sum of squares for the treatment and posttests interaction (variability 
left in a cell after the treatment effects and the posttests effects have 
been subtracted out). 

SSInteraction ** SSGells - SSP - SSB* tween treatments 
» 16,416.7 - 1,623.3 - 14,636.5 
- 156.4 

9. Sum of squares within treatment groups (the sum of the variabilities of 
scores within each treatment group from the treatment group mean). 

SSWithin treatment groups ** £ X 2 - ^^Al) 2 - . . , - (£ ^A4) 2 



« 82,605 - (150) 2 - (492) 2 

~ 6 



« 4,233.5 



10. Sum of squares for the interaction between posttests and the effects of 
levels of ability within treatments (the variability within each treatment 
group after the effects of posttests and the effects of levels within treat- 
ments have been subtracted out). 

SSInteraction of posttests and levels within treatments « 

SSWithin treatment groups - SSP - SS Levels within treatments - SSInteraction. 

- 4,233.5 - 1,623*8 - 2,404.0 - 156.4 

- 49.3 



Table 4 

S/SSBSEL Table Model (for k treatments, j .posttests, and p«kra total levels 

where m is the number of levels within each treatment) 

Source df 

Between Levels p-1 

Between treatments k-1 
(Error between) Levels within 

treatments p-k 

Within Levels d(H) 

Posttests j-1 

Interaction: 

(Treatment x post (k-1) (j-1) — (Use Error within to test this 
test§) MS) 

Interaction: 

(Posttests x Levels 
' within treatments^ 



(Error within) 



— (use Error between to test this MS) 



— (Use Error within to test this MS) 



Total 



N-l 



16 



TABLE V 

SUMMARY TABLE FOR THIS EXAMPLE 













F @ 


Source 


df 


SS 


MS 


F 


.05 


Between Levels 


11 


17,040.5 








Between treatments 
Levels within 


3 


14,636.5 


4,878.8 


16.24 


4.06 


treatments 
[error (between)] 


8 


2,404.0 


300.5 






Within Levels 


12 


1,829.5 








Posttests 
Treatment x 


1 


1,623.8 


1,623.8 


- 




Posttests 

Posttests x Levels 


3 


156.4 


52.1 


8.54 


4.06 


w. Treatment 
[error (within)] 


8 


49.3 


6.1 








23 


18,870.0 









Findings: 



(1) The null hypothesis 



« 0 is rejected. 



(2) The null hypothesis y.. * 0 is rejected. 

(1) The research hypothesis that the four 

instructional treatments have differential 
effects on the over-all performance of the 
students is supported by the data of this 
experiment. 



(2) The research hypothesis that the four 
treatments result in different rates of 
forgetting is supported by these data. 



Conclusions : 



Related Findings: 

1. Location of significant differences. 



By location the significant differences between means 
the researcher can determine which of the four treatments 
had the greater effect on the over-all performance of the 
students. The following conditions hold in this experiment: 

i) the maximum probability of making a type-I 
error (<x) is set at .05* 

ii) the planned contrasts are not orthogonal* 

iii) the error rate (#) has an experiment base* 

iv) contrasts with other than the control group 
are to be made* 

v) the number of comparisons c is 6* 

vi) the number of treatments k is 4* 

vii) the number of comparisons* c =6* is equal to 
k (k - l)/2* and 



viii) only pair-wise contrasts are to be made. 

Based upon the above conditions and the schema recom- 
mended by Hopkins and Chadbourn (4) for making c multiple 
comparisons among k treatment means* the Tukey (b) method 
described by Winer (5) may be applied to locate significant 
differences between means. 

The Tukey (b) method uses a studentized range statis- 





X 



s 



MMSE/n 



tic defined by 



where n is the number of observations in each X. The 
studentized range statistic is the quotient obtained by 
dividing the difference between the largest and smallest 
treatment means (the range of the treatment means) by the 
within group variance (often referred to as the error mean 
square, MSE) of the ANOVA procedure over n. 

The Tukey (b) test is a compromise between the Newman- 
Keuls (6) procedure and the more conservative Tukey (a) 
procedure ( i • e • , more conservative in terms of Keeping the 
type-I error small for a given alpha) . The q-critical of 
the Tukey (b) method is given by 

- « <*' f) + % - « (r * f) 

*c " 2 

where k * number of treatments (i.e., the range of the number 

of steps between means), f = degrees of freedom for USE, and 

r = number of steps between two means on an ordered scale 

which are being compared. The number of steps r between two 

means X. and X. is j - i + 1 where j and i are the rank order 
3 1 

of the means being compared. The symbol q ^ (k, f) desig- 
nates the 99th percentile point on the q distribution,. Tables 
of the studentized range statistic q distribution containing 
critical values of q x _ (k, f) are given in Table B.4 in 

Winer (3). Critical values of a ( r * are obtained 

from the same tables by setting r equal to the range. 
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If the observed student ized range statistic q , is larger 

obs 

than q , the difference between the means is significant. 

The following calculations demonstrate a comparison 
between the means of treatment A1 and A2: 



find q . s 
obs 






25.0 

6 


*A2 = 


4il = 68.8 
6 


n = 


6 


MSE = 


300.5 


n ss 


68.8 - 25.0 


q obs 


if 300. 5/6 


find q : 
c 





= 6.19 



Placing the mean scores in rank order we have. 



a A1 

25.0 



A3 

33.3 



A2 

68.8 



a A4 

82.3 



Hence k = 4 

r = 3 -1 + 1 = 3 
f = 8 



with q (4# 8) * 4.53 



and q ^ (3, 8) = 4.04 



Thus 



4*53.+ .,4« 0 4 » 4.29 
2 
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Since the observed q statistic exceeds the critical value , 
the difference between the means is significant. Pair-wise 
comparisons of other treatments are made in a similar manner. 

2m Slopes of Retention Curves . 

The fact that there is a statistically significant 
treatment x repeated performance measure interaction indi- 
cates that the slopes of the retention curves for the treat- 
ment groups are different. To determine which treatments 
result in decreased rate of forgetting (i.e., improve 
retention) * the slopes of the retention curves may be 
plotted as shown in Figure 5 based upon the data in Table VI. 



TABLE VI 





Immediate 


Delayed 


Treatment A1 


37 


13 


Treatment A2 


75 


64 


Treatment A3 


42 


19 


Treatment A4 


85 


80 



Note: The values used in Table VI are means 

calculated from the data shown on 
page 13. 



3 



60 



Mean 

Test Scores 



50 - 



40 - 



30 - 



20 - 



10 



A3 a 



A1 X ^ 
X 



\ 



X 



\ 



X X 



X 



X 



X 



Immediate Delayed 

Posttests 



Figure 5 



The slopes of the retention curves of treatments A1 
and A3 are essentially the same. Therefore, treatment A3 
offers no advantage over treatment A1 in terms of improvement 
of retention. However, the slopes of A2 and A4 are not as 
steep as A1 and A3 and, hence, represent an improvement in 
retention. 

The slope of A4 being less than the slope of A2 says 
that the rate of forgetting of A4 is less than that 
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of A2 • That is, A4 results in greater improvement 
of retention than any of the other three treat- 
ments. 

3. Estimation of effects : 

Effect of A1 = = X_. - X m = 25.0 - = -26.6 

A1 A1 T 24 

Effect of A2 = « A2 =68.8 - 51.6 =17.2 

Effect of A3 = & = 33.3 - 51.6 = -18.3 

Effect of A4 = O' = 82.3 - 51.6 = 31.7 

A4 

Interaction effect associated with treatment A1 and 
immediate posttest s 

V A1-I = *A1-I ~ *1 *A1 + *T 

= 36.7 - 59.3 - 25.0 + 51.6 

= 4.0 



4. Tests for assumptions of homogeneity of variance : 

(a) The variance "levels within treatments" is a 
pooling of the variances of the level means from the treat- 
ment mean within treatment A1 ; plus the variance of the 
level means from the treatment means within treatment A2 , 
etc. The variances of the level means within the various 
treatments must be homogeneous to have a valid test of the 
treatment effects. This assumption is tested by finding 



o 

ERIC 
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the variances of the level means about each treatment mean 
and testing the homogeneity of these variances by an F-test 



For this example research: 



(EX 



SS of levels within A1 ~ 



Al-Hl 



) 



(EX 



Al-Ml 



) 



SS of levels within A2 = 



SS of levels within A3 = 



SS of levels within A4 = 



I>1S of levels within Al ” 



n Al-Hl 


n Al-Ml 




< SX A1-L1 )2 (EX A1 )2 




n Al-Ll 


n Al 




_ (66) 2 , (55) 2 , 


(29) 2 


(150) 2 _ 


2 2 


2 


6 


_ (165 ) 2 + (144) 2 


+ (104) 2 


(413) 2 


2 2 


2 


6 


(68) 2 j (73) 2 L 


(41) 2 


(182) 2 _ 



,>2 



2 2 
2 



<X89}“ (171)“ (134) 

' 1 ■ ■ ■ ■ + + •» ■ - - 



(494) 



= 786.3 



!®i = lii = 180.5 
df 2 



MS of levels within A2 

MS of levels within A3 

MS of levels within A4 
480.1 



= 960 * 3 
2 

=r 296 - 3 
2 

786.3 



= 480.1 
= 148.1 
= 393.1 



max 148.1 



■ 3.24 



The significance of this ratio is tested by entering the 
F “table for the df of the numerator and the df of the 
denominator-— in this case 2,2 df. 
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F critical with 2.2 df at .05 level = 19.0 
max 

*rhe assumption of homogeneity of the level within 
treatment variances is supported by these data. 

(b) The variance "interaction of posttests and levels 
within treatments” is a pooling of the variance of the scores 
in each treatment group about the treatment mean after the 
effects of posttests and the effects of levels within treat- 
ments have been subtracted out. These variances for each 
treatment group must he homogeneous to have valid tests of 
the posttests mean square and the treatment x posttests mean 
square* This assumption is tested as follows: For each 

treatment, the sum of squares for cells is determined, and 
the SS for posttests and the SS for levels within treatments 
is subtracted out to find the variance of the interaction of 
posttests and levels within treatments for that treatment. 
The variances found for each treatment are then compared for 
homogeneity by the F-test. For this example research: 



For treatment Al: 

2 

SS Ceils 38 (45)^ + (40)^ •** (25) + (21) + ... - ^ “ 1182 

(110) 2 . (40) 2 (150) 2 _ c 

SS Posttests * — + — ■ — - — : 816.6 



3 

2 



6 

2 



..... (66)* (55)* , (29) 

SS Levels within Al = + + 



2 



(150) 



= 361 



2 



2 



2 



6 
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SS of interaction between posttests and levels within treat- 
ments for treatment A1 = 1182 - 816.6 - 361 = 4.4 

The sum of squares of the "posttests by levels within 
treatments" interaction may be calculated for the other treat 
ments in a similar manner. 

A2 = 3.2 

A3 = 41.4 

A4 = 3.3 

These sum of squares are converted into variances by 
dividing by the appropriate degrees of freedom — in this 
example the product of (number of posttests - 1) times 
(number of levels in ai treatment - 1) =2 • 

Variance of Al = 2.20 



The assumption of homogeneity of the posttests x levels 
within treatments variances is supported by the data of this 
experiment. 



Variance of A2 = 1.60 
Variance of A3 = 20.70 
Variance of A4 » 1.65 



P 



max 




F critical at 2,2 df .05 = 19 
max 
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